Universal Access To All Knowledge
Home Donate | Store | Blog | FAQ | Jobs | Volunteer Positions | Contact | Bios | Forums | Projects | Terms, Privacy, & Copyright
Search: Advanced Search
Anonymous User (login or join us)
Upload

Reply to this post | Go Back
View Post [edit]

Poster: Administrator, Curator, or StaffBrad Leblanc Date: Oct 14, 2005 2:36am
Forum: etree Subject: Uploaders - New Policy Re: FFP Files

Please note, as of today we now require all FLAC filesets to include a separate text file containing fingerprint information (FFP). This file helps identify the signature of the original files sent to archive.org without requiring someone to download the whole recording (invaluable to db.etree admins). Even if you are including an MD5 file, we still require a separate FFP.

Please let me know if you have any questions about this. We will update our upload instructions to reflect this change. Thanks for your understanding!

-Brad

This post was modified by Brad Leblanc on 2005-10-14 09:36:10

Reply to this post
Reply [edit]

Poster: xtifr Date: Oct 12, 2005 6:44am
Forum: etree Subject: Re: Uploaders - New Policy Re: FFP Files

Does this mean that db.etree will be picking up records directly from the Archive? Awesome if so!

If I could whip up a small script to allow verifying an upload from FFPs without using a separate MD5 file--a script which would even detect missing files--would the Archive be interested?

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or StaffBrad Leblanc Date: Oct 12, 2005 8:03am
Forum: etree Subject: Re: Uploaders - New Policy Re: FFP Files

Does this mean that db.etree will be picking up records directly from the Archive? Awesome if so!

Not yet, although Mark Goldey's request did prompt this new policy.

If I could whip up a small script to allow verifying an upload from FFPs without using a separate MD5 file--a script which would even detect missing files--would the Archive be interested?

Yes! Please send me a copy - bleblanc at you know the domain.

-Brad

Reply to this post
Reply [edit]

Poster: xtifr Date: Oct 13, 2005 10:07am
Forum: etree Subject: Re: Uploaders - New Policy Re: FFP Files

> > If I could whip up a small script...

> Yes! Please send me a copy...

Hi, thought I would have time to put this together last night, but life happened. The weekend may be better for me.

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or StaffBrad Leblanc Date: Oct 19, 2005 4:41am
Forum: etree Subject: Re: Uploaders - New Policy Re: FFP Files

No worries, when you get a chance. Thanks!

Reply to this post
Reply [edit]

Poster: Jhudson844 Date: Oct 12, 2005 12:40pm
Forum: etree Subject: Re: Uploaders - New Policy Re: FFP Files

"This file helps identify the signature of the original files sent to archive.org without requiring someone to download the whole recording"

You can still check the integrity of a single file with md5 summer. Example you download only d1t01

if all the files fail md5 sum except for d1t01, then d1t01 passed! Sort of surprised this didn't occur to y'all before my cotribution got frozen without any advanced notice. (I don't have time to read this board every day) I'm at work now and can't do anything about it. Will your md5 checker now be able to read ffp's to ensure the integrity of the whole fileset or will we need to generate both now?

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or StaffBrad Leblanc Date: Oct 12, 2005 11:54pm
Forum: etree Subject: Re: Uploaders - New Policy Re: FFP Files

Hi Joel,

You can still check the integrity of a single file with md5 summer. Example you download only d1t01

Yes, but MD5's will fail if the FLAC file has ID tags created or updated down the road, which make them less ideal than a FFP for this. Eventually, I'm hoping we can auto-tag the FLAC files using metaflac in the same way we tag the lossy files. Immediately, this will cause a problem with the MD5's that were originally included. I'm not entirely sure how we should deal with that. For starters we're going to make FFP's required, the immediate benefit being that db.etree.org Sources In Circ admins have them available to file in their database - whereas in the past we have allowed FLAC filesets through with no FFP or MD5.

Will your md5 checker now be able to read ffp's to ensure the integrity of the whole fileset or will we need to generate both now?

There's a pending feature request to get FFP files to be used for verification/auto-email that's been open for years now, it's just low on the priority list of the engineers (I wish I knew how to play with this stuff). xtifr's script will help shorten the time it takes them to get it done, my fingers are crossed that this will be enough to get the ball rolling.

(I don't have time to read this board every day) I'm at work now and can't do anything about it.

That's no problem. If it takes a day or 2 to get us the FFP, we can wait. We acknowledge there will be an adjustment period while people get used to having to include this file from now on. I'm sorry this will slow them down in the next couple days, but we need the benefits. All uploads frozen for this reason will be combined by an email from a curator letting you know why it was frozen.

-Brad

This post was modified by Brad Leblanc on 2005-10-13 06:54:18

Reply to this post
Reply [edit]

Poster: Jhudson844 Date: Oct 13, 2005 1:59am
Forum: etree Subject: Re: Uploaders - New Policy Re: FFP Files

"Yes, but MD5's will fail if the FLAC file has ID tags created or updated down the road, which make them less ideal than a FFP for this. Eventually, I'm hoping we can auto-tag the FLAC files using metaflac in the same way we tag the lossy files. "

I don't seem to recall giving permission for the flac files I've uploaded to be "tagged" or modified in any way

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or StaffBrad Leblanc Date: Oct 13, 2005 2:41am
Forum: etree Subject: Re: Tagging FLAC files

So you don't want the artist or song name to be included with the file? Why? I thought there was no downside to this, and the artists would appreciate it. Please elaborate on the problem. The audio is unchanged...

It would basically add this info to all FLAC files.

Artist: (pulled from artist name you import)
Title: (pulled from song title in File Options)
Track Number: (auto generated based on filenames)
Album: (combination of show date and venue)

What is the concern? Why do we need permission to make sure information about the performance and artist travels with the files here?

Thanks for bringing it up though, this is an issue I obviously had not considered to this point, so I'd like to know what's up.

-Brad

This post was modified by Brad Leblanc on 2005-10-13 09:41:55

Reply to this post
Reply [edit]

Poster: Forkbeard Date: Oct 13, 2005 4:58am
Forum: etree Subject: Re: Tagging FLAC files

Maybe it's the principle that the taper ought to retain control - the same way some tapers don't want lossy conversions.

From my limited perspective, I would welcome pre-tagged FLACs. Tagging is tedious. Auto-tagging might help proliferate lossless formats as well.

Reply to this post
Reply [edit]

Poster: xtifr Date: Oct 13, 2005 5:48am
Forum: etree Subject: Re: Uploaders - New Policy Re: FFP Files

Tags don't affect the audio portion of the file, which is the portion you have some small claim to copyright over. The Flac compression, however, is simply machine translation, and you don't have a copyright on the individual bits of the result (although you still have a partial copyright on the bits derived from your original audio signal). So it's possible your permission wouldn't matter. It's also possible that it would count as "fair use". Copyright law has some fuzzy edges.

Of course, it's possible that the Archive would be willing to respect such wishes if they were strong enough, even if there were no legal basis for such a request. But if you feel that strongly about this paticular issue, please let me know, so I can add you to my personal "never download or listen to anything taped by this asshole" list. :)

Reply to this post
Reply [edit]

Poster: hoyt Date: Oct 13, 2005 11:48am
Forum: etree Subject: Re: Uploaders - New Policy Re: FFP Files

<>

Oops, so if I just uploaded a FLAC show, with MD5s and then "described" the files by putting in information (I usually just leave it blank), the MD5s now fail?

--hoyt

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or StaffBrad Leblanc Date: Oct 13, 2005 11:04pm
Forum: etree Subject: Re: Tagging FLAC files

Not related - we don't tag FLAC files yet so "File Options" does not cause this yet.

This post was modified by Brad Leblanc on 2005-10-14 06:04:40

Reply to this post
Reply [edit]

Poster: Jhudson844 Date: Oct 17, 2005 2:00pm
Forum: etree Subject: Re: Tagging FLAC files

"Will your md5 checker now be able to read ffp's to ensure the integrity of the whole fileset or will we need to generate both now?

There's a pending feature request to get FFP files to be used for verification/auto-email that's been open for years now, it's just low on the priority list of the engineers (I wish I knew how to play with this stuff). xtifr's script will help shorten the time it takes them to get it done, my fingers are crossed that this will be enough to get the ball rolling."

So they can't get that working, and a year and a half later they still haven't figured out a way to allow streaming/lossy copies of older shows but now we're being inconvenienced immediately by having to generate 2 checksums just because they haven't figured out how to get the md5 checker to check ffps and just possibly at some great mythical point in the future they might figure out a way to implement metaflac. I have the upmost confidence in that ever happening.

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or Staffsuspect Date: Oct 19, 2005 10:32pm
Forum: etree Subject: Re: Tagging FLAC files

So they can't get that working, and a year and a half later they still haven't figured out a way to allow streaming/lossy copies of older shows but now we're being inconvenienced immediately by having to generate 2 checksums just because they haven't figured out how to get the md5 checker to check ffps and just possibly at some great mythical point in the future they might figure out a way to implement metaflac.

I like to think of it more as growing pains. This thing is *still* free to use and we appreciate the patience of all of our patrons. There are things that have been waiting in the wings since the LMA's inception and they still haven't shown up for me, such as a new categorical listing of shows, but I have faith that one day, they will come. They have built it, we have come. Now we just wait to see what new stuff can be tossed in the mix. In the meantime, we do what we have to do to fill our (shared) need to fulfill our live music hobby.

Also, ffp's and md5's aren't all *that* time consuming to create. They do take an extra 2 or 3 minutes depending on how many files you have but I was including both with flac shows before it was required by the LMA. It's just good practice. The more ways we can prove that a file is legitimate and pure, the better, I say.

best,
...tom

This post was modified by suspect on 2005-10-20 05:32:43